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ABSTRACT 

The relation between videotape-based observer 
judgements of attention to task and paper-and-pencil measures of 
academic performance was investigated in this study. Forty-five Grade 

pupils engaged in an arithmetic computation task and were 
video-taped for ten consecutive school days. The tapes were then 
independently viewed by three observers, and pupils were rated on 
their observable attention to task. Significant correlations on the 
order of .50 (p less than .01) were obtained between attention to 
task ratings and academic performance. An intraclass correlation of 
.93 was obtained as an estimate of inter-observer reliability. The 
data supported the use of indirect observational methods in assessing 
school performance. (Author/MC) 
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Abstract 



The relation betx>reen video tape based observer judgments 
of attention to task and paper and pencil measures of academic 
performance xvas investigated. Forty-five fourth-grade pupils 
engaged in an arithmetic computation task were video taped 
for ten consecutive school days. Tapes were independently 
viewed by three observers and pupils were rated on attention 
to task. Significant correlations on the order of ,50 
(p < .01) ^^fere obtained between attention to task ratings and 
academic performance. An intraclass correlation of .93 was 
obtained as an estimate of inter-observer reliability. Data 
supports the use of indirect observational methods in assess- 
ing school performance. 
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During recent years the assessment of pupil performance 
has came to occupy a central position among efforts directed 
at the improvement of educational practice. Schemes that 
have emerged for assessing student performance classify as 
either (a) direct measures of school performance, e. g , , the 
continuous measure of pupil response rate (Gassholt, 1970; 
Lovitt Curtiss, 1969) or (b) indirect measures of academic 
performance, e.g, , the degree to which pupils are observed 
attending, to task (Attwell , Orpet , g jMeyers, 1967; Ferritor, 
Buckholdt, Hamblin, Pt Smith, 1971; Hall, Lund, e, Jackson, 
1968) • However, advantages and disadvantages accrue to each 
approach. 

It is true that direct measures are sensitive to ongoing 
changes in pupil performance and that such measures have the 
added benefit of providing students with immediate feedback. 
Yet direct measurement procedures can present real hardships 
for teachers in the form of logistical problems. Since the 
educational setting typically'^provides the teacher with a 
multitude of tasks, it is often difficult for her to see the 
relative merit in maintaining a comprehensive system of con- 
tinuous measurement. Such plans are therefore often discarded 
after an enthusiastic but brief trial period. 

Indirect schemes for measuring pupil performance relieve 
the teacher of some time constraints imposed by research. Often 
trained observers located in the classroom handle the recording 
of both teacher and pupil behavior and no special modification 
of class schedule is required. The procedure typically involves 
an observer selecting a target behavior and recording the fre- 
quency of occurrence of that behavior through a time -sampling 
process. The observer is, however, severely restricted in the 
number and kind of behaviors that can be counted. Further, 
since .observers do not always agree, most observational arrange- 
ments necessitate the presence of a second observer so that an 
index o£ inter-observer agreement can be obtained. The approach 
is generally restricted to the recording of small "bits*' of 
pupil behavior, since it is difficult to obtain observer agree- 
ment on more complex behavioral sequences. Perhaps the most 
damaging criticism of the '"observer present'* technique is the 
concern over the lack of evidence involving the validity of 
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the measures obtained. The inference that attention to task 
behaviors are accompanied by a corresponding increase in aca- 
demic performance has received only limited attention. 

Recently, hov/ever, modern technology has provided tech- 
niques such as time lapse photography, motion pictures, and 
video tape recording that offer advantages over the indirect 
procedure of maintaining trained observers in the classroom* 
Flanders (1967), for example, has used audio tapes to train 
observers, and a major attempt has been made to record and ^ 
translate the entire spectrum of teacher-pupil interaction 
via video tape (Adams fi Biddle, 1970) • By us ing video tape 
equipment, it is a relatively simple task to record individual 
or group rates of ^^attention to task/' and overcome most of 
the limitations imposed on observers in the classroom. 

The present study was undertaken to investigate two 
problems of indirectly measuring and evaluating pupil perform- 
ance using a video tape recording device/ First, to evaluate 
the efficacy of a stop-action video tape technique for reliably 
measuring ongoing behavioral events, in particular, the extent 
to which groups of pupils attend to task in a regular class- 
room. Second, to determine the degree to which measures of 
attention to task recorded on video tape xvere correlated xv^ith 
paper and pencil measures of the quality and quantity of asCa- 
demic performance simultaneously obtained during taping sessions 
Such a determination examined the relation between indirect 
measures of performance and actual academic output c 
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Sub j ects 

Forty- five fourth- grade pupils, 25 males and 20 females, 
from Hudson Elementary School, Tempe, Arizona, served as sub- 
jects in the stuc' The most recent Stanford Achievement 
Series'-Arithmeti Com: Uwation subtest scores were obtained 
for each S v/ith the exception of one. Arithmetic computation 
ability for that S was estimated by her teacher (T) . Subtest, 
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scores v;.ere reported in the form of grade level equivalents 
and were used to estimate the current level of arithmetic com- 
putation ability for each S. Scores for Ss on this achievement 
computation measure ranged from 4.6 to 2.2 with a mean of 3.47, 
Pupils were selected as S^s for this study on the bases of 
whether their desks were included in the optimal camera angle 
for each area, and whether they had obtained parental permis- 
sion to be video taped. 

Apparatus 

A Sony AV-3400 portable record/playback videorecorder hav- 
ing a stop-action capability and a Sony AVC-3400 video camera 
equipped with a Soligor f/1.8 wide angle lens was used to 
record and play back observation sessions. A tone pulse system 
was used to record an audio tone of one -second duration at 30- 
S6iCond intervals on the video tape sound track. This system 
consisted of a rotary- cam microswitch pulse generator connected 
to a DC relay. Every 30 seconds the pulse generator provided 
a one-second flow of AC current to the DC relay producing a ' 
distinct buzz . The system was isolated with a low impedence 
microphone in a ten-inch by five-inch box, so that the buzz 
could not be heard by S^s during taping sessions. Another loT\r 
impedence microphone was used to record normal classroom noise 
level. Both microphones were connected to a Shure inicrophone 
mixer, which allowed the classroom noise and the buzz signal 
to be recorded simultaneously on the video tape sound track. 

The videorecorder, tone generating system, microphone 
mixer,, and a Sony recorder and camera powerpack were mounted 
on a small wheeled cart equipped with a two -plug extensioiT cord.. 
Thus, power for all components was provided by energizing a 
single-line cord. The equipment was arranged so that the pulse 
generator could be activated independently from other components. 
The video camera was mounted on a standard tripod and the entire 
system was easily moved to different locations in the classroom.^ 



^Appreciation is extended to Ronald D. Zellner for the 
development of the tone pulse system used in this study. 
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Proce dure 

The observational setting consisted of a large open class- 
room containing 125 pupils grouped. in four seating areas of 
approximately 31 pupils each. For taping and identification 
purposes these areas were numbered Area 1, Area 2, Area 3, and 
Area 4. Twelve Ss were taped in Area 1, 13 in Area 2, and 10 
each in Areas 3 and 4. In all, 45 pupils were video taped and 
served as Ss in the study. During the afternoon the pupils 
ivere normally seated in heterogeneous ability groups, and the 
arrangement was not disturbed for taping .sess ions . 

Taping sessions consisted of 10-minute arithmetic seat- 
work sessions. Each pupil in the entire area was given an 
identical worksheet of 80 arithmetic problems that were taken 
at random from the T»s handbook (Eichology, O'Daffer, Brumfiel, 
§ Shanks, 196 4). Types of problems included 2-, 3-, and 4~ 
digit addition, subtraction, multiplication, and division 
problems. T accompanied her distribution of seatwork materials 
by saying: ''It is time for an arithmetic practice session; 
please put your name and the date at the top of the page. 
Don't begin work until I ask you to do so." 

Mien the teacher told the pupils to begin work, E started 
the video camera, stopwatch, and pulse generator simultaneously. 
Thus, the initial buzz on the tape sound track occurred 30 
seconds after Ss had begun work. This insured that 20 buzzer 
tones would be recorded on the sound track during the 10-minute 
session. At the end of the 10-minute period, E signalled the 
teacher to have the pupils stop work, and simultaneously turned 
off the video camera. The arithmetic materials were collected 
and E moved to the next area. 

This procedure was repeated with each of the four class- 
room groups. Ten different ivork sheets containing equivalent 
arithmetic computation problems were used over the ten-day 
taping period. Each day 125 pupils in the room engaged in 
the same arithmetic material for a 10 -minute period, and the 
45 pupils who were seated within camera range served as Ss 
in the srudy . 
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Data Recording 

• — — iJL 

Tv/0 kinds of behavioral measures, (a) attention to task, 
and Cb) academic performance v/ere used to assess the merits 
of the video tape technique and the relation between tlie two 
•measures. The academic performance data for each S took two 
forms: (a) total number of problems completed, and (b) total 
number of problems completed correctly. IVork performed by Ss 
during taping sefssions v/as hand scored for these two performance 
measures. Data on the attention to task measure were taken from 
the video tapes by three trained observers (Os) . 

Three graduate students were trained as Os in a single 
45-minute training session. The Os were told that they were 
going to view tapes of children working in a classroom, and 
that their job x>ras to judge whether the pupils were attending 
or not attending to tSisk at the end of 30-second intervals. 
The following definition of attention to task behavior was 
then given to Os : VJhenever the child is fully seated at his 
desk, looking at his paper, and holding his pencil in his hand, 
he is considered^ to be attending to task. Any negation of 
these criteria such as looking or walking around the room^ or 
not holding the pencil will be considered non-attention. 

Os were provided data recording sheets corresponding to 
seating charts of the areas that were taped. The data sheets 
for the four different areas showed only seating position of 
S s , no names were given. Using the data sheets as a guide, 
Os were showm a sho,rt video tape of each area in order to 
orient them to Ss ' seating arrangements . 

After Os indicated they could relate the tapes to the data 
sheets, they were given instructions regarding actual observa- 
tion procedures as follows: 

After you have threaded the tape, place the control 
lev>3r in the 'forward' position. At the end of a 30- 
second interval you will hear a distinct buzz. At 
. that time place the still mode control lever in the" 
'still' position. This will freeze the picture and 
allow you to judge each child appearing on your 
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data sheet for each area. At each judgment interval 
record either an 'X' indicating attention to task 
behavior or an '0' indicating a non-attention to 
task behavior for each child. After you have recorded 
your judgments, return the still mode lever to the 
'off position and the picture will begin moving 
again. Are there any questions? Remember, there 
will be 20 buzzer tones heard during each 10 -minute 
session,, so you should stop the tape 20 times and 
make 20 judgments on each child. 

Follov/ing a careful reading of the instructions, Os were 
given approximately 10 minutes of training on operating the 
videorecorder . V/lien each 0 expressed condifence in the observa 
tion procedures, the training session ended. 



Results 



In order to determine the reliability of the video tape 
technique for classroom observation, an intraclass correlation 
coefficient was computed. The coefficient was obtained by 
summing the number of "on task'^ judgments made by each observer 
on all Ss and using these sums in the computation of the coef - 
ficient. The intraclass index for the three 6s engaged in the 
study was .93 indicating a high degree of reliability among 
observer ratings of pupil attending behaviors. 

To examine the relationship between observer ratings and 
academic performance , an arithmetic average was taken on the 
total number of on-task judgments made by the three Os for 
each S over the 10 -day period. These mean values were then 
correlated with the 10-day sum, for each S, on the following 
three measures: (a) total number of problems completed, 
Cb) total number of problems correct, and (c) grade level 
equivalent on the Stanford Achievement Tes t^Arithmetic Com- 
putation subtest- The correlation matrix for these four* vari- 
ables is shown in Table 1. 



TABLE 1 

Correlation jiatrix o£ the Four Relevant 
Variables for All 45 Ss 



1 Total problems 

completed 

2 Total problems 

correct 

3 Observer ratings 

4 Stanford Achievement 

scores 



1.000 

0.8125 
0.5107 
0. 2551 



1.000 



0.4881 



0. 5383 



1.000 



0 . 20 60 



1.000 



The values in Table 1 represent the obtained intercorre - 
lations of the four variables under cons ideration for all 45 
S s . it is noteworthy that while Stanford Achievement scores 
and 0 ratings have similar correlations with problems correct, 
0 ratings exhibit a much stronger association with the quantity 
of work completed. 

Arithmetic computation ability logically accounts for a 
portion of the association between academic performance meas- 
ures and attention to task ratings, and since this effect would 
confound the association between performance and attention with 
another factor, i.e ., arithmetic ability , these effects were 
held constant by applying the partial correlation technique to 
the data used in computing the intercorrelation matrix illus- 
trated in Table 1. The correlation between total problems 
completed and attention to task, with ability partialled out 
was .4843 (as compared to the regular Pearson r of .5107). 
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The partial correlation for total correct probleiaG and atten- 
tion to task v/as .4576 (Pearson r = .4881). These partial cor- 
relation coefficients were significantly different from zero 
(P < .01) . 

To determine the effect of differential arithmetic compu- 
tation ability on the association betv/een academic performance 
and attention to task , arithmetic computation scores were used 
to divide Ss into high (grade level equivalent of 4.0 and above), 
average (3.00 to 3.99), and low (2.99 and below) ability groups. 
The correlation matrices for this analysis are presented in 
Table 2. 

It should be noted that the Stanford Achievement scores 
have been excluded from the analysis in Table 2 because these 
scores were the basis for the formation of the ability groups. 
The correlations between the measures of academic performance 
and attention to task were highest for the h igh ability group 
(. 5592 and .4759), and progressively lov/er for the othe*^ groups 
(.49 41 and .4628 for the average group and . 3254 and .4316 for 
the 1 ow group). However, only the values for the avera ge 
group were significantly different from, zero (p < .01) , perhaps 
due to the small sample size of the high and 3;ow groups. Also, 
it is of interest that the correlation betwee:f" the number of 
problems completed and the number of problems correct was .99 27 
for the high group, .769 3 for the average group, and .7287 for 
the lox>r group. 

Finally, to ascertain the effect of sex differences, all 
45\Ss were dichotomized into male and female groups. The cor- 
relation matrices for all four variables, i.e ., both academic 
performance measures, 0 ratings of attention to task, and 
Stanford Achievement scores, are shown for these two groups 
in Table 3. 

An examination of the values in Table 3 reveals that the 
correlations between the two acadiem.ic performance measures and 
attention ratings were slightly higher for females than for 
males. When the effect of arithmetic computation ability was 
partialled out, the correlations between total problems com- 
pleted and problems correct with ratings of attention were 
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.4982 and .S086 respectively for the male group. The partial 
correlations for the female group were .5724 between problems 
completed and attention, and .5203 between problems accurate • 
and attention. The partial correlations for the male group 
were significantly different from zero (p < .01) as were those 
for the female group (p < .05). No significant difference 
between groups (p > .25) was found using Fisher's transforma- 
tion to z (Guilford, 1965). 

In order to show differences in magnitude and variability, 
between all groups in the analysis of all four variables, the 
means and standard deviations of the four variables are shown 
in Table 4. . 

Of special interest in Table 4 are the mean differences 
on all four variables between the three groups (high, average 
and low) . In general, the mean values followed a trend from 
highest magnitude in the high group, somewhat less for the 
average group, and least for the low group. The only excep- 
tion occurred with the 0 rating variable between the high and 
average group. These values were nearly identical for both 
groups (high group = 159.14, average group =161.57). 

Discussion 



One of the major purposes of this study was to determine 
the empirical relationship between indirect measures of academic 
performance, i.e ., attention to task behavior, and actual 
academic output. The data indicate that such indirect meas- 
ures are significantly related to both the quantity and quality 
of school performance. Thus, these findings confirm those of 
Cobb (19 72) and Lahaderne (1968).- 

The results were also of practical significance to those 
investigators who use indirect measures of academic performance 
as dependent variables in experimental research . As a result 
of the data obtained in this study and by others (Cobb, 1972; 
Lahaderne, 196 8), the inference that attention to task behaviors 
are related to academic performance is strengthened. In fact, 
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in this study, the attention to task measure accounted for 
more of the variance in one of the academic performance meas- 
ures (total problems completed) than did the pupil's achieve- 
ment test scores in arithmetic computation (approximately 25 1 
versus approximately 6%), and accounted for nearly the same 
amount of variance in the total problems correct variable 
(approximately 23% versus approximately 28%). This was prob- 
ably due to the fact that quality of work reflected primarily 
an ability factor, while quantity reflected more of a motiva- 
tional component. Thus , it was not surprising that Stanford 
Achievement scores were more strongly associated with quality 
than with quantity. V/hat was important was that attention 
ratings were nearly as related to quality as the Stanford 
scores, and much more related to quantity. 

Several other of the obtained correlations v/ere also of 
interest. The entire group of Ss was divided into three sub- 
groups based on amount of arithmetic computation ability. This 
was done to see \/hether bright children needed to attend as 
much to perform the same amount o£ work as less able pupils. 
The results were equivocal. Ifliile the correlations for the 
high ability group were highest for attention to task, problems 
completed, and problems correct of the three ability groups, 
the average group obtained a higher mean number of attention 
to task ratings (161.57 versus 159.14) yet performed at a lower 
rate both quantitatively and qualitatively . The finding is an 
interesting one and needs to be investigated experimentally. 
One possible interpretation is that "bright'* children do not 
need to attend more than other children to produce the academic 
output expected o£ them. 

Since sex differences in school performance seem'to occur 
with regularity, Ss were divided into male and female groups 
to determine if such differences were present. In general, it 
has been found that male pupils have relatively greater dif- 
ficulty in school, both academically and socially (Kagan, 1971) . 
Since the variables recorded in this :tudy involved both ca- 
de mi c and social behaviors, it seemed plausible that the asso- 
ciation would be different for boys than for girls. Although 
the female group had a higher rate of absolute performance than 
the male group, the association between the relevant variables 
was nearly identical for both sexes. 
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Efficacy of the video tape technique > The use of video 
tape recordings of pupil behaviors in itself does not offer 
anything new to scientific investigation. Audio tapes, stili 
cameras, and motion pictures have been used quite often as 
investigatory tools. In this study, however, video tape record- 
ings were used as a vehicle to improve the reliability and 
validity of more standard "observer present** time-sampling 
tcdmiques. 

With the standard time-sampling technique, several 
observers record the behavior of children one at a time at pre- 
specified intervals. Thus, observers are recording the behaviors 
of different children at different times. Through the use of 
video tapes, and their **still** mode function, each observer 
rates the same behavior of the same pupil at the same time. 
Hence, not only are the reliability and validity of the observ- 
ers' judgments improved, but the length of the time interval 
is more flexible. For example, it would be nearly impossible 
for an observer using a standard time-sampling procedure to . 
record the behavior of a classroom of pupils every five seconds. 
Using the video tape technique, however, the problem is reduced. 

In addition, video tapes are a permanent record of behav- 
ior, and as such permit flexibility in data analysis. Post hoc 
analyses can be applied to determine the context of the behav- 
ioral variables being rated, to determine if important sequenc- 
ing effects have been overlooked, or to analyze yet another set 
of variables from the same tape. Further, since each observer 
is rating virtually identical instances of behavior, more 
pox^rerful techniques for estimating observer reliability are 
applicable, such as the intraclass correlation coefficient. 

Since this study was concerned with the relation between 
attending . behaviors and academic performance , as represented 
by an arithmetic computation task, it seems necessary to inves - 
tigate the association as it relates to other subject matter 
areas and tasks. For example, what is the relation between 
attention and reading proficiency? Is attending, as defined 
in this study, an important variable when the child is engaged 
in a manipulative task such as a science experiment? Can impor- 
tant attending behaviors be exhibited in \\rays other than those 
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described iierein? All of these are inportant questions that 
are amenable to systematic investigation. Secondly, since 
causal relationships cannot be ascertained from the correla- 
tional methods used in this study, experimental investigations 
that manipulate attending rates need to be performed. Some 
studies of this type have been attempted (Becker, Madsen, 
Arnold, Thomas, 1967; Bushell, Wrobel, Michaelis, 1968, 
^^^^y £i 5 1068) but more work remains. Finally, the 
technique used in this study is highly applicable to the 
^•field'^ type studies carried out in the area of child develop- 
ment. Experimenters employing variables like aggression, 
dependency, imitation, etc. which are conducted in natural 
settings such as nursery school playgrounds should find the 
technique extremely useful in the observation and recording 
of child behavior. 
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